{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finding the best advertisement banner using bandits\n",
    "\n",
    "In this section, let's understand how to find the best advertisement banner using the\n",
    "bandits. Suppose, we are running a website and we have five different banners for a single\n",
    "advertisement that is been showed on our website and say we want to figure out which\n",
    "advertisement banner is most liked by the user.\n",
    "\n",
    "We can frame this problem as a multi-armed bandit problem. The five advertisement\n",
    "banners represent the five arms of the bandit and we assign +1 reward if the user clicks the\n",
    "advertisement and 0 reward if the user does not click the advertisement. So, to find out\n",
    "which advertisement banner is most clicked by the user, that is which advertisement\n",
    "banner can give us a maximum reward, we can use various exploration strategies. In this\n",
    "section, let's just use an epsilon-greedy method to understand the best advertisement\n",
    "banner.\n",
    "\n",
    "First, let us import the necessary libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "%matplotlib inline\n",
    "plt.style.use('ggplot')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a dataset\n",
    "\n",
    "Now, let's create a dataset. We generate a dataset with five columns denoting the five\n",
    "advertisement banners and we generate 100000 rows where the values in the row will be\n",
    "either o or 1 indicating that whether the advertisement banner has been clicked (1) or not\n",
    "clicked (0) by the user:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame()\n",
    "for i in range(5):\n",
    "    df['Banner_type_'+str(i)] = np.random.randint(0,2,100000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the first few rows of our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Banner_type_0</th>\n",
       "      <th>Banner_type_1</th>\n",
       "      <th>Banner_type_2</th>\n",
       "      <th>Banner_type_3</th>\n",
       "      <th>Banner_type_4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Banner_type_0  Banner_type_1  Banner_type_2  Banner_type_3  Banner_type_4\n",
       "0              0              1              0              1              1\n",
       "1              1              1              1              0              0\n",
       "2              0              1              1              0              0\n",
       "3              1              1              1              0              0\n",
       "4              0              1              0              0              0"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can observe we have the 5 advertisement banners (0 to 4) and\n",
    "the rows consist of value 0 or 1 indicating that whether the banner has been clicked (0) or not clicked (1). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Initialize the variables\n",
    "\n",
    "Now, let's initialize some of the important variables:\n",
    "\n",
    "Set the number of iterations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_iterations = 100000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the number of banners:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_banner = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize count for storing the number of times, the banner was clicked:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "count = np.zeros(num_banner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize sum_rewards for storing the sum of rewards obtained from each banner: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "sum_rewards = np.zeros(num_banner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initialize Q for storing the mean reward of each banner:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "Q = np.zeros(num_banner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the list for storing the selected banners:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "banner_selected = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the epsilon-greedy method\n",
    "\n",
    "Now, let's define the epsilon-greedy method. We generate a random value from a uniform\n",
    "distribution. If the random value is less than epsilon, then we select the random banner else\n",
    "we select the best banner which has a maximum average reward:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def epsilon_greedy_policy(epsilon):\n",
    "    \n",
    "    if np.random.uniform(0,1) < epsilon:\n",
    "        return  np.random.choice(num_banner)\n",
    "    else:\n",
    "        return np.argmax(Q)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the bandit test\n",
    "\n",
    "Now, we run the epsilon-greedy policy to understand which is the best advertisement\n",
    "banner:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#for each iteration\n",
    "for i in range(num_iterations):\n",
    "    \n",
    "    #select the banner using the epsilon-greedy policy\n",
    "    banner = epsilon_greedy_policy(0.5)\n",
    "    \n",
    "    #get the reward of the banner\n",
    "    reward = df.values[i, banner]\n",
    "    \n",
    "    #increment the counter\n",
    "    count[banner] += 1\n",
    "    \n",
    "    #store the sum of rewards\n",
    "    sum_rewards[banner]+=reward\n",
    "    \n",
    "    #compute the average reward\n",
    "    Q[banner] = sum_rewards[banner]/count[banner]\n",
    "    \n",
    "    #store the banner to the banner selected list\n",
    "    banner_selected.append(banner)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After all the rounds, we can select the best banner as the one which has the maximum\n",
    "average reward:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The optimal banner is banner 4\n"
     ]
    }
   ],
   "source": [
    "print( 'The optimal banner is banner {}'.format(np.argmax(Q)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also plot and see which banner is selected most of the times:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEJCAYAAACzPdE9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3df1DTd57H8WfCDxWjmATFlWIrKnPiykBNt2BbRMzVndLZc9TaaWu72rq2g6vTuuP4o3uyO3t69CiFcotnf1i6ve72x1nr7c3tXO9YRpkr6zYWwva0rVDrdTlBJIliEIuQ3B+2aV1BUb8mAq/HX8kn32++7893El58vp9vvl9TMBgMIiIico3MkS5ARESGBgWKiIgYQoEiIiKGUKCIiIghFCgiImIIBYqIiBgiOtIFRNqxY8ciXYKIyKAyadKkPts1QhEREUOEZYTS3d1NYWEhPT099Pb2kpWVxdKlS6moqODQoUPExcUBsHr1am655RaCwSCVlZXU19czYsQICgoKSElJAWDv3r3s3r0bgEWLFpGbmwvAkSNHqKiooLu7m8zMTFasWIHJZApH90REhDAFSkxMDIWFhYwcOZKenh62bNlCRkYGAA8//DBZWVkXLF9fX09rayvl5eU0Njby8ssvs23bNvx+P7t27aKoqAiAjRs34nA4sFgsvPTSSzz++ONMnz6dv//7v8ftdpOZmRmO7omICGE65GUymRg5ciQAvb299Pb2XnL0cODAAXJycjCZTKSmptLZ2YnP58PtdpOeno7FYsFisZCeno7b7cbn89HV1UVqaiomk4mcnBxcLlc4uiYiIl8J26R8IBBgw4YNtLa2smDBAqZPn85//ud/8sYbb7Br1y6++93v8tBDDxETE4PX6yUhISG0rt1ux+v14vV6sdvtoXabzdZn+9fL96WqqoqqqioAioqKLtiOiIhcvbAFitlspri4mM7OTp599lm++OILHnzwQcaNG0dPTw8vvPAC//qv/8qSJUvo63qV/Y1oTCZTn8v3x+l04nQ6Q8/b29uvvDMiIsPYDXOW1+jRo0lLS8PtdmO1WjGZTMTExDBv3jyampqA8yOMb/+h93g8WK1WbDYbHo8n1O71erFardjt9gvaPR4PNpstfJ0SEZHwBEpHRwednZ3A+TO+PvroI5KSkvD5fAAEg0FcLhfJyckAOBwOampqCAaDHD58mLi4OKxWKxkZGTQ0NOD3+/H7/TQ0NJCRkYHVamXUqFEcPnyYYDBITU0NDocjHF0TEZGvhOWQl8/no6KigkAgQDAYJDs7m9mzZ/Pzn/+cjo4OAG6++WZWrVoFQGZmJnV1daxdu5bY2FgKCgoAsFgsLF68mE2bNgGwZMkSLBYLACtXrmT79u10d3eTkZGhM7xERMLMNNxvsKVfyovItTB/XBLpEq6LwIyf9PvaDTOHIiIiQ5MCRUREDKFAERERQyhQRETEEAoUERExhAJFREQMoUARERFDKFBERMQQChQRETGEAkVERAyhQBEREUMoUERExBAKFBERMYQCRUREDKFAERERQ4TtnvIig917v22JdAnXxYIffCfSJcgQoRGKiIgYQoEiIiKGUKCIiIghFCgiImIIBYqIiBgiLGd5dXd3U1hYSE9PD729vWRlZbF06VLa2tooKyvD7/czZcoU1qxZQ3R0NOfOneOXv/wlR44cYcyYMTz55JNMmDABgHfffZfq6mrMZjMrVqwgIyMDALfbTWVlJYFAgPnz57Nw4cJwdE1ERL4SlhFKTEwMhYWFFBcX8w//8A+43W4OHz7M66+/Tn5+PuXl5YwePZrq6moAqqurGT16NP/4j/9Ifn4+v/71rwFobm6mtraW5557jqeffpqdO3cSCAQIBALs3LmTzZs3U1payvvvv09zc3M4uiYiIl8JS6CYTCZGjhwJQG9vL729vZhMJg4ePEhWVhYAubm5uFwuAA4cOEBubi4AWVlZ/M///A/BYBCXy8WcOXOIiYlhwoQJTJw4kaamJpqampg4cSKJiYlER0czZ86c0HuJiEh4hO2HjYFAgA0bNtDa2sqCBQtITEwkLi6OqKgoAGw2G16vFwCv14vdbgcgKiqKuLg4Tp8+jdfrZfr06aH3/PY6Xy//9ePGxsY+66iqqqKqqgqAoqIiEhISjO+sDFFD84eN+g5cG2+kC7hOruZzEbZAMZvNFBcX09nZybPPPsv//d//9btsMBi8qM1kMvXZfqnl++J0OnE6naHn7e3tlytdZEjTd+DaDNUzmy71uZg0aVKf7WHfF6NHjyYtLY3GxkbOnDlDb28vcH5UYrPZgPMjDI/HA5w/RHbmzBksFssF7d9e5y/bPR4PVqs1jL0SEZGwBEpHRwednZ3A+TO+PvroI5KSkpg5cyb79+8HYO/evTgcDgBmz57N3r17Adi/fz8zZ87EZDLhcDiora3l3LlztLW10dLSwrRp05g6dSotLS20tbXR09NDbW1t6L1ERCQ8wnLIy+fzUVFRQSAQIBgMkp2dzezZs7npppsoKyvjzTffZMqUKeTl5QGQl5fHL3/5S9asWYPFYuHJJ58EIDk5mezsbNatW4fZbOaxxx7DbD6fiY8++ihbt24lEAgwb948kpOTw9E1ERH5iinY38TEMHHs2LFIlyCDhK42LH0xf1wS6RKui8CMn/T72g0zhyIiIkOTAkVERAyhQBEREUMoUERExBAKFBERMYQCRUREDKFAERERQyhQRETEEAoUERExhAJFREQMoUARERFDKFBERMQQChQRETGEAkVERAyhQBEREUMoUERExBAKFBERMYQCRUREDKFAERERQyhQRETEEAoUERExRHQ4NtLe3k5FRQUnT57EZDLhdDq55557ePvtt/n973/P2LFjAXjggQe49dZbAXj33Xeprq7GbDazYsUKMjIyAHC73VRWVhIIBJg/fz4LFy4EoK2tjbKyMvx+P1OmTGHNmjVER4eleyIiQpgCJSoqiocffpiUlBS6urrYuHEj6enpAOTn5/ODH/zgguWbm5upra3lueeew+fz8Ytf/ILnn38egJ07d/LTn/4Uu93Opk2bcDgc3HTTTbz++uvk5+dzxx138OKLL1JdXc3dd98dju6JiAhhOuRltVpJSUkBYNSoUSQlJeH1evtd3uVyMWfOHGJiYpgwYQITJ06kqamJpqYmJk6cSGJiItHR0cyZMweXy0UwGOTgwYNkZWUBkJubi8vlCkfXRETkK2E/JtTW1sbnn3/OtGnT+OSTT3jvvfeoqakhJSWFRx55BIvFgtfrZfr06aF1bDZbKIDsdnuo3W6309jYyOnTp4mLiyMqKuqi5f9SVVUVVVVVABQVFZGQkHC9uipDTkukC7gu9B24Nv3/azy4Xc3nIqyBcvbsWUpKSli+fDlxcXHcfffdLFmyBIC33nqL1157jYKCAoLBYJ/r99VuMpmuqAan04nT6Qw9b29vv6L1RYYafQeuzVA9s+lSn4tJkyb12R62fdHT00NJSQl33XUXt99+OwDjxo3DbDZjNpuZP38+n332GXB+5OHxeELrer1ebDbbRe0ejwer1cqYMWM4c+YMvb29FywvIiLhE5ZACQaD7Nixg6SkJO69995Qu8/nCz3+4IMPSE5OBsDhcFBbW8u5c+doa2ujpaWFadOmMXXqVFpaWmhra6Onp4fa2locDgcmk4mZM2eyf/9+APbu3YvD4QhH10RE5CthOeT16aefUlNTw+TJk1m/fj1w/hTh999/n6NHj2IymRg/fjyrVq0CIDk5mezsbNatW4fZbOaxxx7DbD6ffY8++ihbt24lEAgwb968UAg99NBDlJWV8eabbzJlyhTy8vLC0TUREfmKKdjfhMUwcezYsUiXIIPEe78dmpPyC37wnUiXMKiZPy6JdAnXRWDGT/p9LeJzKCIiMrQpUERExBAKFBERMYQCRUREDKFAERERQyhQRETEEAoUERExhAJFREQMoUARERFDKFBERMQQChQRETGEAkVERAyhQBEREUMoUERExBAKFBERMYQCRUREDKFAERERQyhQRETEEAoUERExhAJFREQMoUARERFDKFBERMQQ0QNd8A9/+APZ2dkXte/fv5+srKxLrtve3k5FRQUnT57EZDLhdDq555578Pv9lJaWcuLECcaPH89TTz2FxWIhGAxSWVlJfX09I0aMoKCggJSUFAD27t3L7t27AVi0aBG5ubkAHDlyhIqKCrq7u8nMzGTFihWYTKaBdk9ERK7RgEcoO3bs6LP9hRdeuOy6UVFRPPzww5SWlrJ161bee+89mpub2bNnD7NmzaK8vJxZs2axZ88eAOrr62ltbaW8vJxVq1bx8ssvA+D3+9m1axfbtm1j27Zt7Nq1C7/fD8BLL73E448/Tnl5Oa2trbjd7oF2TUREDHDZQDl+/DjHjx8nEAjQ1tYWen78+HH+9Kc/ERsbe9mNWK3W0Ahj1KhRJCUl4fV6cblczJ07F4C5c+ficrkAOHDgADk5OZhMJlJTU+ns7MTn8+F2u0lPT8disWCxWEhPT8ftduPz+ejq6iI1NRWTyUROTk7ovUREJDwue8hr7dq1ocdr1qy54LVx48Zx3333XdEG29ra+Pzzz5k2bRqnTp3CarUC50Ono6MDAK/XS0JCQmgdu92O1+vF6/Vit9tD7Tabrc/2r5fvS1VVFVVVVQAUFRVdsB2RS2uJdAHXhb4D16bvvzSD39V8Li4bKG+99RYAhYWF/PznP7/yqr7l7NmzlJSUsHz5cuLi4vpdLhgMXtTW33yIyWTqc/n+OJ1OnE5n6Hl7e/uA1xUZivQduDZD9cymS30uJk2a1Gf7gPfFtYZJT08PJSUl3HXXXdx+++0AxMfH4/P5APD5fIwdOxY4P8L4dmc8Hg9WqxWbzYbH4wm1e71erFYrdrv9gnaPx4PNZrumekVE5MoM+CyvtrY23njjDY4ePcrZs2cveO2f/umfLrluMBhkx44dJCUlce+994baHQ4H+/btY+HChezbt4/bbrst1P4f//Ef3HHHHTQ2NhIXF4fVaiUjI4M33ngjNBHf0NDAgw8+iMViYdSoURw+fJjp06dTU1PD97///QHvBBERuXYDDpTnn3+exMREHnnkEUaMGHFFG/n000+pqalh8uTJrF+/HoAHHniAhQsXUlpaSnV1NQkJCaxbtw6AzMxM6urqWLt2LbGxsRQUFABgsVhYvHgxmzZtAmDJkiVYLBYAVq5cyfbt2+nu7iYjI4PMzMwrqlFERK6NKTjACYgf/vCHVFZWYjYPrSOGx44di3QJMki899uhOSm/4AffiXQJg5r545JIl3BdBGb8pN/XrnkOZcaMGRw9evSKixIRkeFhwIe8xo8fz9atW/ne977HuHHjLnjt/vvvN7wwEREZXAYcKF9++SWzZ8+mt7f3gjOqRERE4AoC5euJcRERkb4MOFCOHz/e72uJiYmGFCMiIoPXgAPl25dg+Utf/5peRESGrwEHyl+GxsmTJ/mXf/kXZsyYYXhRIiIy+Fz1j0rGjRvH8uXL+c1vfmNkPSIiMkhd068Ujx07xpdffmlULSIiMogN+JDXli1bLrji75dffsmf//xnlixZcl0KExGRwWXAgZKXl3fB85EjR3LzzTfzne/osg0iInIFgfL1vdtFRET6MuBA6enpYffu3dTU1ODz+bBareTk5LBo0SKiowf8NiIiMkQNOAlef/11PvvsM370ox8xfvx4Tpw4wTvvvMOZM2dYvnz5dSxRREQGgwEHyv79+ykuLmbMmDHA+csXT5kyhfXr1ytQRERk4KcNX8l920VEZPgZ8AglOzubZ555hiVLlpCQkEB7ezvvvPMOWVlZ17M+EREZJAYcKMuWLeOdd95h586d+Hw+bDYbd9xxB4sXL76e9YmIyCBx2UD55JNPOHDgAMuWLeP++++/4GZar7/+OkeOHCE1NfW6FikiIje+y86hvPvuu6SlpfX52ne/+112795teFEiIjL4XDZQjh49SkZGRp+vzZo1i88//9zwokREZPC57CGvrq4uenp6iI2Nvei13t5eurq6LruR7du3U1dXR3x8PCUlJQC8/fbb/P73v2fs2LEAPPDAA9x6663A+VFRdXU1ZrOZFStWhALN7XZTWVlJIBBg/vz5LFy4EIC2tjbKysrw+/1MmTKFNWvW6MeWIiJhdtkRSlJSEg0NDX2+1tDQQFJS0mU3kpuby+bNmy9qz8/Pp7i4mOLi4lCYNDc3U1tby3PPPcfTTz/Nzp07CQQCBAIBdu7cyebNmyktLeX999+nubkZOD+Xk5+fT3l5OaNHj6a6uvqyNYmIiLEuGyj5+fm8+OKL/PGPfyQQCAAQCAT44x//yEsvvUR+fv5lN5KWlobFYhlQQS6Xizlz5hATE8OECROYOHEiTU1NNDU1MXHiRBITE4mOjmbOnDm4XC6CwSAHDx4Mnb6cm5uLy+Ua0LZERMQ4lz0udOedd3Ly5EkqKio4d+4cY8eOpaOjg9jYWO677z7uvPPOq974e++9R01NDSkpKTzyyCNYLBa8Xi/Tp08PLWOz2fB6vQDY7fZQu91up7GxkdOnTxMXF0dUVNRFy/elqqqKqqoqAIqKikhISLjq+mW4aYl0AdeFvgPXpv+/NoPb1XwuBjTRcO+995KXl8fhw4fx+/1YLBZSU1OJi4u74g1+7e677w7dS+Wtt97itddeo6CgoN9f5PfV/u37swyU0+nE6XSGnre3t1/xe4gMJfoOXJtrukvhDexSn4tJkyb12T7gmeu4uLh+z/a6GuPGjQs9nj9/Ps888wxwfuTh8XhCr3m9Xmw2G8AF7R6PB6vVypgxYzhz5gy9vb1ERUVdsLyIiIRPxMLV5/OFHn/wwQckJycD4HA4qK2t5dy5c7S1tdHS0sK0adOYOnUqLS0ttLW10dPTQ21tLQ6HA5PJxMyZM9m/fz8Ae/fuxeFwRKRPIiLDWVjOrS0rK+PQoUOcPn2aJ554gqVLl3Lw4EGOHj2KyWRi/PjxrFq1CoDk5GSys7NZt24dZrOZxx57DLP5fO49+uijbN26lUAgwLx580Ih9NBDD1FWVsabb77JlClTLrq7pIiIXH+m4DC/jPCxY8ciXYIMEu/9dmhOyi/4gW7jfS3MH5dEuoTrIjDjJ/2+1t8cylCdTxIRkTBToIiIiCEUKCIiYggFioiIGEKBIiIihlCgiIiIIRQoIiJiCAWKiIgYQoEiIiKGUKCIiIghdJ/cPrSsXxnpEq6L7xS/HOkSRGQI0whFREQMoUARERFDKFBERMQQmkORS1r+qz9EuoTr4tUfZke6BJEhRyMUERExhAJFREQMoUARERFDKFBERMQQChQRETGEAkVERAwRltOGt2/fTl1dHfHx8ZSUlADg9/spLS3lxIkTjB8/nqeeegqLxUIwGKSyspL6+npGjBhBQUEBKSkpAOzdu5fdu3cDsGjRInJzcwE4cuQIFRUVdHd3k5mZyYoVKzCZTOHomoiIfCUsI5Tc3Fw2b958QduePXuYNWsW5eXlzJo1iz179gBQX19Pa2sr5eXlrFq1ipdfPn/9Kb/fz65du9i2bRvbtm1j165d+P1+AF566SUef/xxysvLaW1txe12h6NbIiLyLWEZoaSlpdHW1nZBm8vl4mc/+xkAc+fO5Wc/+xnLli3jwIED5OTkYDKZSE1NpbOzE5/Px8GDB0lPT8disQCQnp6O2+1m5syZdHV1kZqaCkBOTg4ul4vMzMxwdE1kWCovL490CdfF2rVrI13CoBaxX8qfOnUKq9UKgNVqpaOjAwCv10tCQkJoObvdjtfrxev1YrfbQ+02m63P9q+X709VVRVVVVUAFBUVXbCtr7VcW9duWH31dbi6un0xND8Z+lx842r2Rf9/bQa3q9kXN9ylV4LB4EVt/c2HmEymPpe/FKfTidPpDD1vb2+/sgIHseHU18vRvviG9sU3rmZfDNUzmy61LyZNmtRne8T2RXx8PD6fDwCfz8fYsWOB8yOMb3fE4/FgtVqx2Wx4PJ5Qu9frxWq1YrfbL2j3eDzYbLYw9UJERL4WsUBxOBzs27cPgH379nHbbbeF2mtqaggGgxw+fJi4uDisVisZGRk0NDTg9/vx+/00NDSQkZGB1Wpl1KhRHD58mGAwSE1NDQ6HI1LdEhEZtsJyyKusrIxDhw5x+vRpnnjiCZYuXcrChQspLS2lurqahIQE1q1bB0BmZiZ1dXWsXbuW2NhYCgoKALBYLCxevJhNmzYBsGTJktAE/cqVK9m+fTvd3d1kZGRoQl5EJALCEihPPvlkn+1btmy5qM1kMrFyZd+34M3LyyMvL++i9qlTp4Z+3yIiIpExVOeTREQkzBQoIiJiCAWKiIgYQoEiIiKGUKCIiIghFCgiImIIBYqIiBhCgSIiIoZQoIiIiCEUKCIiYggFioiIGEKBIiIihlCgiIiIIRQoIiJiCAWKiIgYQoEiIiKGUKCIiIghFCgiImIIBYqIiBhCgSIiIoZQoIiIiCGiI13A6tWrGTlyJGazmaioKIqKivD7/ZSWlnLixAnGjx/PU089hcViIRgMUllZSX19PSNGjKCgoICUlBQA9u7dy+7duwFYtGgRubm5EeyViMjwE/FAASgsLGTs2LGh53v27GHWrFksXLiQPXv2sGfPHpYtW0Z9fT2tra2Ul5fT2NjIyy+/zLZt2/D7/ezatYuioiIANm7ciMPhwGKxRKpLIiLDzg15yMvlcjF37lwA5s6di8vlAuDAgQPk5ORgMplITU2ls7MTn8+H2+0mPT0di8WCxWIhPT0dt9sdyS6IiAw7N8QIZevWrQD89V//NU6nk1OnTmG1WgGwWq10dHQA4PV6SUhICK1nt9vxer14vV7sdnuo3Waz4fV6w9gDERGJeKD84he/wGazcerUKf7u7/6OSZMm9btsMBi8qM1kMvW5bH/tVVVVVFVVAVBUVHRBQH2tZSCFD0J99XW4urp9MTQ/GfpcfONq9sVQ/df1avZFxAPFZrMBEB8fz2233UZTUxPx8fH4fD6sVis+ny80v2K322lvbw+t6/F4sFqt2Gw2Dh06FGr3er2kpaX1uT2n04nT6Qw9//b7DXXDqa+Xo33xDe2Lb1zNvrgh5w0McKl90d8//hHdF2fPnqWrqyv0+E9/+hOTJ0/G4XCwb98+APbt28dtt90GgMPhoKamhmAwyOHDh4mLi8NqtZKRkUFDQwN+vx+/309DQwMZGRkR65eIyHAU0RHKqVOnePbZZwHo7e3lzjvvJCMjg6lTp1JaWkp1dTUJCQmsW7cOgMzMTOrq6li7di2xsbEUFBQAYLFYWLx4MZs2bQJgyZIlOsNLRCTMIhooiYmJFBcXX9Q+ZswYtmzZclG7yWRi5cqVfb5XXl4eeXl5htcoIiIDM1QP/4mISJgpUERExBAKFBERMYQCRUREDKFAERERQyhQRETEEAoUERExhAJFREQMoUARERFDKFBERMQQChQRETGEAkVERAyhQBEREUMoUERExBAKFBERMYQCRUREDKFAERERQyhQRETEEAoUERExhAJFREQMoUARERFDKFBERMQQ0ZEuwEhut5vKykoCgQDz589n4cKFkS5JRGTYGDIjlEAgwM6dO9m8eTOlpaW8//77NDc3R7osEZFhY8gESlNTExMnTiQxMZHo6GjmzJmDy+WKdFkiIsOGKRgMBiNdhBH279+P2+3miSeeAKCmpobGxkYee+yxC5arqqqiqqoKgKKiorDXKSIyVA2ZEUpfuWgymS5qczqdFBUV3TBhsnHjxkiXcMPQvviG9sU3tC++caPviyETKHa7HY/HE3ru8XiwWq0RrEhEZHgZMoEydepUWlpaaGtro6enh9raWhwOR6TLEhEZNobMacNRUVE8+uijbN26lUAgwLx580hOTo50WZfldDojXcINQ/viG9oX39C++MaNvi+GzKS8iIhE1pA55CUiIpGlQBEREUMMmTmUwUiXijlv+/bt1NXVER8fT0lJSaTLiaj29nYqKio4efIkJpMJp9PJPffcE+myIqK7u5vCwkJ6enro7e0lKyuLpUuXRrqsiAkEAmzcuBGbzXbDnj6sQImQry8V89Of/hS73c6mTZtwOBzcdNNNkS4t7HJzc/n+979PRUVFpEuJuKioKB5++GFSUlLo6upi48aNpKenD8vPRUxMDIWFhYwcOZKenh62bNlCRkYGqampkS4tIn73u9+RlJREV1dXpEvplw55RYguFfONtLQ0LBZLpMu4IVitVlJSUgAYNWoUSUlJeL3eCFcVGSaTiZEjRwLQ29tLb29vnz9WHg48Hg91dXXMnz8/0qVckkYoEeL1erHb7aHndrudxsbGCFYkN5q2tjY+//xzpk2bFulSIiYQCLBhwwZaW1tZsGAB06dPj3RJEfHqq6+ybNmyG3p0AhqhRMxALxUjw9PZs2cpKSlh+fLlxMXFRbqciDGbzRQXF7Njxw4+++wzvvjii0iXFHYffvgh8fHxoZHrjUwjlAjRpWKkPz09PZSUlHDXXXdx++23R7qcG8Lo0aNJS0vD7XYzefLkSJcTVp9++ikHDhygvr6e7u5uurq6KC8vZ+3atZEu7SIKlAj59qVibDYbtbW1N+QHRMIrGAyyY8cOkpKSuPfeeyNdTkR1dHQQFRXF6NGj6e7u5qOPPuJv/uZvIl1W2D344IM8+OCDABw8eJB/+7d/u2H/VihQImSwXirmeigrK+PQoUOcPn2aJ554gqVLl5KXlxfpsiLi008/paamhsmTJ7N+/XoAHnjgAW699dYIVxZ+Pp+PiooKAoEAwWCQ7OxsZs+eHemy5BJ06RURETGEJuVFRMQQChQRETGEAkVERAyhQBEREUMoUERExBAKFBERMYR+hyJigNWrV3Py5EnMZjPR0dGkpqbyox/9iISEhEiXJhI2GqGIGGTDhg388z//My+88ALx8fG88sorkS7pAr29vZEuQYY4jVBEDBYbG0tWVha/+tWvAKirq+PNN9/k+PHjxMXFMW/evNCNotra2vjxj39MQUEBb731Ft3d3eTn57No0SIA3n77bZqbm4mNjeWDDz4gISGB1atXM3XqVOD8VatfeeUVPv74Y1GSVAYAAAKOSURBVEaOHEl+fn7ohlxvv/02f/7zn4mJieHDDz/kkUceueEvfy6Dm0YoIgb78ssvqa2tDV1qfcSIEfz4xz+msrKSjRs38l//9V988MEHF6zzySef8Pzzz/O3f/u37Nq1i+bm5tBrH374IXPmzOHVV1/F4XCERj6BQIBnnnmGW265hRdeeIEtW7bwu9/9DrfbHVr3wIEDZGVlUVlZyV133RWG3stwphGKiEGKi4uJiori7NmzxMfH8/TTTwMwc+bM0DI333wzd9xxB4cOHeJ73/teqP2+++4jNjaWW265hZtvvpn//d//Dd2l8a/+6q9C1/LKycnh3//93wH47LPP6OjoYMmSJQAkJiYyf/58amtrycjIACA1NTW0ndjY2Ou8B2S4U6CIGGT9+vWkp6cTCARwuVwUFhZSWlrKiRMn+M1vfsMXX3xBT08PPT09ZGVlXbDuuHHjQo9HjBjB2bNnQ8/j4+NDj2NjYzl37hy9vb2cOHECn8/H8uXLQ68HAgFmzJgRev7tm7iJXG8KFBGDmc1mbr/9dl588UU++eQTfv3rX7NgwQI2bdpEbGwsr776Kh0dHde8nYSEBCZMmEB5ebkBVYtcO82hiBgsGAzicrno7OwkKSmJrq4uLBYLsbGxNDU18d///d+GbGfatGmMGjWKPXv20N3dTSAQ4IsvvqCpqcmQ9xe5UhqhiBjkmWeewWw2YzKZGD9+PKtXryY5OZmVK1fy2muv8corr5CWlkZ2djadnZ3XvD2z2cyGDRt47bXXWL16NT09PUyaNIn777/fgN6IXDndD0VERAyhQ14iImIIBYqIiBhCgSIiIoZQoIiIiCEUKCIiYggFioiIGEKBIiIihlCgiIiIIf4fY7oCVy/J+GAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ax = sns.countplot(banner_selected)\n",
    "ax.set(xlabel='Banner', ylabel='Count')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, we learned how to find the best advertisement banner by framing our problem as a multi-armed bandit problem"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
